Dialogue Strategy Redesign with Reliability Measures

This paper reports on user evaluation for a spoken dialogue system and the subsequent redesign of the dialogue strategy. An automatic telephone directory system developed at Philips Research, PADIS, was evaluated using both objective metrics and subjective evaluation techniques. Duration and the occurrence of misunderstanding leading tòdead-ends', were found to be the key factors that kept users from using the system. Therefore, the dialogue strategy was redesigned to avoid`dead-end' misunderstandings. Prompts were redesigned for speed. Reliability measures provided by the speech-recognition component are used to limit the number and length of veriication questions. Tasks with a higher risk or cost need to be veriied more accurately .